Globally sparse PLS regression

Partial least squares (PLS) regression combines dimensionality reduction and prediction using a latent variable model. It provides better predictive ability than principle component analysis by taking into account both the independent and response variables in the dimension reduction procedure. However, PLS suffers from over-fitting problems for few samples but many variables. We formulate a new criterion for sparse PLS by adding a structured sparsity constraint to the global SIMPLS optimization. The constraint is a sparsity-inducing norm, which is useful for selecting the important variables shared among all the components. The optimization is solved by an augmented Lagrangian method to obtain the PLS components and to perform variable selection simultaneously. We propose a novel greedy algorithm to overcome the computation difficulties. Experiments demonstrate that our approach to PLS regression attains better performance with fewer selected predictors. Tzu-Yu Liu Electrical Engineering and Computer Science Department, University of Michigan, USA e-mail: [email protected] Laura Trinchera MMIP Departement, AgroParisTech and UMR518 MIA, INRA, France e-mail: [email protected] Arthur Tenenhaus Department of Signal Processing and Electronic Systems, Supélec, France e-mail: [email protected] Dennis Wei Electrical Engineering and Computer Science Department, University of Michigan, USA e-mail: [email protected] Alfred O. Hero Electrical Engineering and Computer Science Department, University of Michigan, USA e-mail: [email protected] 1 To appear in 2013 Springer Verlag book series: "New perspectives in Partial Least Squares and Related Methods"